Use of a convolutional neural network to auto-determine a floor height and floor height elevation of a building

ABSTRACT

A system, apparatus, computer program product, and method use a convolutional neural network to auto-determine a first floor height (FFH) and a FFH elevation (FFE) of a building. The FFH, and FFE of the building are determined with respect to the terrain or surface of the parcel of land on which the building is located. In turn, by knowing the FFH and/or FFE of the building on the parcel, it is possible to use that information while performing a flood risk assessment to a property without requiring a personal inspection of the parcel by a human.

CROSS REFERENCE TO RELATED PATENT APPLICATIONS

The present application is a continuation of U.S. patent applicationSer. No. 16/864,342, filed May 1, 2020, which claims the benefit of theearlier filing date of U.S. provisional patent application 62/842,023,filed May 2, 2019, the entire contents of which being incorporatedherein by reference. The present application also contains subjectmatter related to that disclosed in U.S. Pat. No. 9,639,757, the entirecontents of which being incorporated herein by reference.

BACKGROUND Technical Field

The present disclosure relates to computer-based systems that performimage recognition on images of real properties, especially improvementson the real properties, such as residential buildings. The disclosurealso relates to systems and methods that use the detected objects alongwith other information to identify a floor height and elevation of abuilding.

SUMMARY

According to an aspect of the present disclosure, a computer-basedsystem automatically, accurately, and reliably detects floor heights andelevations of improvements (e.g., buildings, such as houses) on parcelsof land without human inspection of the property. Fusing aspects ofimage processing, artificial intelligence (AI), property characteristicsand spatial and elevation information (such as building footprint,lowest adjacent grade, and land slope) of a specific property, thepresent computer-based system identifies a first floor height (FFH) ofthe improvement with respect to the terrain (or surface) of the parcelof land on which the improvement is located. In turn, by knowing the FFHof the improvement on the parcel, it is possible to improve onconventional methods for estimating flood risk to a property withoutrequiring a personal inspection of the structure by a human. Likewise,having this FFH information in a database for properties in a regionenables the insurance industry and governments to more effectivelyexpand the flood insurance coverage to homeowners who do not presentlyquality, as well as lower financial risks from flood hazards, andincrease recovery speed against flood disasters.

For obtaining flood insurance, homeowners typically are required toconduct a certified analysis to identify a lowest floor elevation ontheir property. This is a manually intensive process that costs between$169 to $2,000 according to massivecert.com, and often is a factor thatcauses homeowners to not obtain flood insurance even though theirproperty is at risk of flood damage. In a traditional survey method, ahomeowner must hire engineers or surveyors to dig in their yard and tofind the lowest spot, and then determine FFH before they can acquireflood insurance. The present disclosure describes a way of determiningFFH without manually inspecting the property and leverages AI technologyapplied to images to obtain FFH from any direction and location alongthe structure's footprint, thus avoiding the effort and expense ofobtaining the information manually. Moreover, because the present systemand method can identify FFH based on image processing and featuredetection, the present disclosure also addresses how to create adatabase of FFHs for properties across the United States, thus greatlysimplifying and lowering the cost thresholds for individuals to applyfor, and obtain flood insurance.

According to one embodiment, a method of detecting a first floor height(FFH) of a first floor of a building relative to a terrain of a parcelof land on which the building is located, the method includes:

obtaining information on a building footprint of the building on theparcel of land;

applying an image of the building to a convolution neural network-based(CNN-based) AI engine that has been trained to identify a first floor ofa building from the image;

analyzing the image with the CNN-based AI engine and determining the FFHof the building;

extracting digital elevation map information of the terrain from adataset for the parcel of land;

converting the FFH of the building to a first floor elevation (FFE) fromthe FFH and the digital elevation map information; and

identifying a part of the building footprint at a lowest adjacent grade(LAG) along the building footprint so as to detect an elevation of theparcel of land and the FFE of the building at the LAG along the buildingfootprint.

In other embodiments, a system and non-tangible computer readable mediumare disclosed for detecting a first floor height (FFH) of a first floorof a building relative to a terrain of a parcel of land on which thebuilding is located.

BRIEF DESCRIPTION OF THE DRAWINGS

The patent or application contains at least one drawing executed incolor. Copies of this patent or patent application publication withcolor drawing(s) will be provided by the Office upon request and paymentof the necessary fee.

A more complete appreciation of the invention and many of the attendantadvantages thereof will be readily obtained as the same becomes betterunderstood by reference to the following detailed description whenconsidered in connection with the accompanying drawings, wherein:

FIG. 1 is a diagram a house having a first floor elevation (FFE) that isbelow a level of flood.

FIG. 2 is a graph of mean damage ratio vs. water depth for various realproperties with differing structures and building types.

FIG. 3 is a table of Federal Emergency Management Agency (FEMA) floodinsurance rates for properties in A* zones based on elevation variancebetween the lowest flood elevation and the base flood elevation (BFE),wherein A* zones are areas subject to inundation by a1-percent-annual-chance flood event.

FIG. 4 is an example of a manually developed building elevationcertification often required for obtaining flood insurance.

FIG. 5 is a National Oceanic and Atmospheric Administration (NOAA) chartthat lists 1,000 yr. storm events.

FIG. 6 is an image illustrating image processing performed by AI engineaccording to the present disclosure in which the AI engine createsbounding boxes around regions of interest (ROI) used to identify objectsin an image.

FIG. 7 is an aerial image of a region with multiple property buildingfootprints individually outlined in the image.

FIGS. 8A, 8B, and 8C are images showing interior and exteriorperspectives of different properties.

FIG. 9 is a diagram showing an arrangement of components of a system andmethod for automatically determining a floor elevation of a realproperty according to the present disclosure.

FIG. 10 is an aerial image of an area with example building footprintswith street segments identified.

FIG. 11 is an aerial image with an outline around a parcel, as well asentrance points, highest adjacent grade (HAG), lowest adjacent grade(LAG), and other points regarding FFH measurements identified.

FIG. 12A is a diagram of related images showing how FFH is shown alongwith a parcel centroid, building centroid, LAG, and front-viewnotations.

FIG. 12B is a training image for FFH relative to slope.

FIG. 12C is a training image for comparing FFE, BFE, and LAG.

FIGS. 13A and 13B are training images showing example real propertieswith convex geometries at respective entrances to the buildings on thereal properties.

FIG. 14 is a training image showing example real properties with concavegeometries at respective entrances to the buildings on the realproperties.

FIG. 15 is an example training image of buildings on land parcels wheregeometries of entrance doors are not coplanar with a front of thebuilding.

FIG. 16 is a training image of buildings with entrances that are near togeometry changes in a middle portion of the respective buildingfootprints.

FIG. 17 is a training image of a buildings with front walkways ofvarying geometries.

FIG. 18 is an overhead training image of a building with a walkway thatapproaches the building.

FIGS. 19A and 19B are training images of packages adjacent to a mainentrance of a building.

FIG. 20 is a training image of cars detected in front of a hotelentrance.

FIG. 21 is a training image of entrance door locations that are detectedby image motion.

FIG. 22 is a plan view of a training image of a building layout thatshows property entrance locations.

FIG. 23 is an aerial image of that shows LAG elevation for examplebuilding as determined by an auto-determination process according to thepresent disclosure.

FIG. 24 is a training image of a building showing a relationship betweena detected ground elevation, FFE and entrance stairs.

FIG. 25 is a training image of a front door object and a heightmeasurement relative to a ground level.

FIG. 26 is a training image of a front door with a detected lock, doorhandle, and door knocker.

FIG. 27 is a training image of a front entrance area of a building withdetected light fixtures, door knocker, and door lock.

FIG. 28 is an example image of night light detection.

FIG. 29 is a training image of a front of a building with detectedhandrails next to the building entrance.

FIG. 30 is a training image of a front view of a building withidentified crawlspaces below the FFE of the building.

FIG. 31 is a training image of a crawlspace area with a detected pillarshowing a height difference between a ground elevation and a FFE.

FIGS. 32A-32C are training images of basement windows and a height fromground level to a basement floor.

FIG. 33 is a training image of an elevated property.

FIG. 34 is a training image of heating and air conditioning appliances.

FIG. 35 is a diagram showing a FFE relative to a lowest (below) floorelevation.

FIG. 36 is an image of a basement and a height between a floor of thebasement and a ceiling of the basement.

FIG. 37 is a training image of an interior of a split level house.

FIG. 38 is a training image of an exterior of a split level house.

FIG. 39 is a chart of unfinished space codes stored in a database fordifferent types of houses.

FIG. 40 is a chart of foundation codes stored in a database fordifferent types of houses.

FIG. 41 is a chart of story codes stored in a database for differenttypes of houses.

FIG. 42 is a diagram showing relative heights and distances between avehicle and a front of a building of a property.

FIG. 43 is an image of detected features in a kitchen and acorresponding plan view of the detected features in an object diagram.

FIG. 44 is a training image of an elevation adjustment from an elevationenabled device by object height.

FIG. 45 is a chart of global sea level rise.

FIG. 46 is a training image of a front of a house with bounding boxes aspart of a semantic segmentation of the image.

FIG. 47 is a dataflow of how different information is fused to obtainFFH/FFE data for respective properties.

FIG. 48 is a data extraction and data analysis network of a computingdevice according to an embodiment.

FIG. 49 is an example of at least one subject image that will be used togenerate a source vector according to an embodiment.

FIG. 50 is a diagram of a data extraction network according to thepresent disclosure.

FIG. 51 is a diagram of a data analysis network according to the presentdisclosure.

FIG. 52 is a diagram showing how components of a source vectors arearranged to correspond with image pixels according to an embodiment.

FIG. 53 is a diagram showing how to identify a property entrance basedon a street facing segment of a building footprint geometry.

FIG. 54 is a chart of sources of FFH information used by a methodaccording to an embodiment.

FIG. 55 is a diagram of computer circuitry according an embodimentaccording to the present disclosure.

FIG. 56 is a flowchart of a process performed according to the presentdisclosure to fuse image processing results, and AI analysis and digitalmap information of terrain to identify the FFH and FFE of a house.

FIGS. 57A and 57B are flowcharts of processes performed for training anAI engine to detect FFH based on images of properties, and then usingthe AI engine for detecting FFHs based on input images of houses.

DETAILED DESCRIPTION

The present disclosure may be embodied as a system, a method, and/or acomputer program product. The computer program product may include acomputer readable storage medium on which computer readable programinstructions are recorded that may cause one or more processors to carryout aspects of the embodiment.

As used herein, an element or step recited in the singular and proceededwith the word “a” or “an” should be understood as not excluding pluralelements or steps, unless such exclusion is explicitly recited.Furthermore, references to “one embodiment” of the present invention arenot intended to be interpreted as excluding the existence of additionalembodiments that also incorporate the recited features.

Elevations are references made with respect to sea level (such as FFEs,lowest floor elevations, and associated measurements). First floorelevations are criteria for providing flood insurance pricing becausethey are related to flood-depth based loss assessment. The concept forFFE for a one-story structure and its relationship to the water depthfrom a flooding event is presented in FIG. 1 . In FIG. 1 , as can beseen, when a sub-floor does not exist (such as basement) in a property,FFEs occur adjacent to the ground elevation, and are the lowest floorelevations in properties. Thus, FFE contains two components: first floorheight (i.e., height of the floor above the ground elevation) and theground elevation. Accordingly, when a flood level surpasses FFE abovethe ground, the first floor of the house floods. A depth of the flood inthe building is directly related to the height of the flood elevationlevel above the FFE.

Flood loss, in an insurance context, can be estimated by using the flooddepth from the FFE and vulnerability curves (mean damage ratio as afunction of flood inundation depth). Example vulnerability curves areshown in FIG. 2 , which plots mean damage ratio (MDR) to water depth infeet for various house structures, with and without basements. As can beseen, the MDR increases with water depth, where the first floor islocated (e.g., whether a basement is included or not), and number offloors. For a given structure, and water depth above the FFE, the MDRcan provide an estimate of flood damage to the building as a product ofthe MDR and water depth. Accordingly, for a $100,000 house, thatexperiences a water depth of 5 feet, with a corresponding MDR of 20%,would have an estimated flood damage of $20,000.

Flood analytic models and insurance companies often take the FFH betweenthe FFE and ground elevation as a model input for flood losscomputations or their rating methods. The sub-floor information (such asbasement) is modeled with parameters (such as “One Story Building withBasement”), implemented by using different vulnerability curves. Forsome others (such as FEMA), the lowest floor elevations are requiredmeasurements for flood insurance policy underwriting. Normally, buildingentrances are constructed at the first floor as main access to thebuilding. Conducting manual elevation surveys on the FFE of structuresand determining the lowest elevations of properties by physicallymeasuring vertical heights of other components of the properties is anexpensive and labor-intensive task.

In the United States, federal flood insurance (NFIP) and its rates areregulated by the relationships between the Base Flood Elevation (BFE), acomputed elevation to which floodwater is anticipated to rise during a100 yr. flood event, and the lowest floor elevations of properties. FIG.3 shows NFIP insurance rates at different depth variance between thelowest floor elevations and the BFE at the location for A* Zones. A*zones present the floodplain with 1% annual chance of flooding (100 yr.flood, and example of a base flood). As presented in FIG. 3 , aninsurance rate can be increased substantially with rise in water depth.As recognized by the present inventors, when water depth above theassociated floor is not appropriately determined, either flood insurancerates are insufficient to ensure the financial health of a floodinsurance program, or they represent overpayment by customers. Incomparison with other hazards, flood is the only peril that requiressuch detailed physical measurements for natural hazard insurance,because a small elevation difference would lead flood water flowing todifferent directions and inundating a different area.

To acquire NFIP flood insurance, homeowners are required to haveconducted on their property a determination of lowest floor elevation.This is performed manually, and a cost of a typical elevationcertificate (EC) ranges from $167 to $2,000 according tomassivecert.com. FIG. 4 shows an exemplary Building ElevationCertificate. There could be over 40 million properties in the US thatare at some level of flood risk, so obtaining an EC for each would beexorbitant, over $12 billion in aggregate.

ECs were one of reasons that only about 50% properties in special floodhazard areas (SFHAs) have flood insurance. The cost of manuallyperforming an evaluation of a property is also very high, and to makesure coverage rates are protected sufficiently by the insurancecompanies, the insurance rates are out of reach for many homeowners.This is best illustrated by recognizing that at the national level,roughly 30% of national properties are at some level of flood risk, butonly 4% of properties have flood insurance policies. As recognized bythe present inventors, developing an effective technology that can lowerthe cost barrier of flood insurance so more people can have their housecovered, is of high interest.

In recent years, severe flood events hit the United States frequentlyand repetitively. FIG. 5 shows the NOAA list of 1,000 yr. storm eventssince 2010. Severe storms occur almost every year. Thus, a huge numberof homes remain at risk and as climate conditions change, and sea levelsrise, this risk is only becoming more amplified. As highlighted byrectangular boxes in FIG. 5 , the 1,000 yr. flood events (such asEllicott City Floods, highlighted in the table) were reoccurring at thesame location as frequently as a two-year period.

Based on the NFIP statistics, about 90% of flood claims have waterdepths less than 3 feet. Therefore, water depth should be much moregranular than 3 feet. In other words, to ensure accuracy of water depthcomputation in flood loss estimation, the technology for determiningfirst floor height or elevations would ideally have the ability tocontrol its vertical accuracy on elevation measurements under severalinches and at the most, less than a single foot. Otherwise, flood lossanalyses and flood insurance pricing may not be reliable.

Smartphones have become one of the important tools in today's life.However, both horizontal and vertical accuracy of the GPS units insmartphones would be in the range of multiple meters, insufficient forthe purpose of floor elevation determination. Therefore, more sufficienttechnology needs to be developed for addressing the problems in thefloor elevation determination.

In recent years, artificial intelligence (AI) has advanced profoundly onabilities to perform object detection on photo images and videos hasmade significant progress. AI technology can automatically detect,recognize, and extract objects from photo images and videos and convertthem into the consumable information for many purposes. FIG. 6 is atraining image with boundary boxes around objects that are detected by atrained AI image processing engine. U.S. Pat. No. 10,438,082, the entirecontents of which being incorporated herein by reference, describes anAI engine that is trained to form boundary boxes around regions ofinterest (ROI) in objects and then have the type of object detected inthat image. As will be discussed later in this disclosure, FIGS. 48-52describe a detailed AI-based engine trained to detect features (e.g.,front doors of houses) in images to ascertain where the FFH is locatedon an improvement (e.g., house) on a parcel of land. In FIG. 6 ,boundary boxes are formed around a streetlight (blue), buildings (red),entrance (purple), windows (dark blue), people (green), and cars (gray).

Because photo images can often contain complex information fusion,reliabilities of the object detections would be difficult to reach 100%.

As recognized by the present inventors, a reliability of detectingspecific property-related objects, including property entrances, doors,doorsteps and other related objects, is an attribute of the presentdisclosure, so that the detected property objects can help to accuratelydetermine floor elevations, which in turn is then directly useful forperforming flood risk assessment and flood insurance rating purposes.

As discussed herein, there are three general categories of informationthat that are fused to form a comprehensive solution to floor elevationdetermination. First, Corelogic's comprehensive national propertycharacteristic database is used, which contains 147 million properties,and covers over 99% of the US population. This database includes landand property characteristics with over 200 data elements andmeasurements on the respective properties. The database has beengeospatially enabled by the property parcels and building footprints.Building footprints define floor area extents, where building wallscould be constructed. Any structure objects (such as entrances, doors,doorsteps, and rooms) of buildings and contents inside buildings havetheir unique correlations to building footprints both geospatially andphysically, and that is the scientific foundation of the property objectdetection in this technological development.

An example of building footprint data available in the database is shownin FIG. 7 , with red-dashed outlines over building footprints, andadjacent streets. Although not shown in FIG. 7 , the database alsoincludes parcel overlay information (like that shown in FIG. 11 ) suchthat a property boundary for each parcel viewed in an arial image arevisible. Moreover, for the 147 million properties, the database includesproperty characteristics, property photos, building layouts, floodinsurance information, multiple listing information and history,geocoded tax, hazard data, assessor maps, estimated cost, and parcel andstructure footprint polygons, as well as other, as described athttps://www.corelogic.com/about-us/data.aspx.

Another source of information is the increasing availability of highresolution and bare-earth elevation datasets with submeter verticalaccuracy, generated by LiDAR (Light Detection and Ranging) technologies.United States Geological Survey (USGS) produces a 1 m digital elevationdataset (DEM), which is being expanded for nationwide coverage(https://pubs.usgs.gov/tm/11/b07/tmll-b7.pdf). According to the presentdisclosure, by obtaining relative heights of exterior and interiorproperty objects above the bare-earth elevation, floor elevations arethen calculated. The present inventors realized that building entrancesare designed as the main access for the structures, and commonlyconnecting to the main floor, called the first floor. If the height ofbottom of the entrance doors or the heights of the top of doorstep abovethe ground are measured, FFEs can be determined by adding those heightsto their ground elevation. If a structure sits on a land slope, firstfloor height would be different at different locations around thestructure because land elevation varies. High resolution elevationdataset and structure building footprint geometries can be used tofacilitating the “FFH to FFE” conversion at the location where FFH ismeasured. The approach can provide a flexibility to allow advancedimagery processing technology to measure FFH from any direction aroundthe structure.

In addition, many billions of high resolution photos and videos fromboth inside and outside properties have been taken with smartphones. Thephotos created from smartphones in close distance ranges can be verydetailed and information rich. The entrances of properties are one ofthe most important components of structures. In addition to the photocollected in Corelogic's database, other images are available ofproperties, such as postings on multiple listing service (MLS), whichincludes photos and building information associated with a parcel'saddress, those available through photogrammetry techniques, and otherswith dimensions of entrances, doorsteps, and rooms. Examples of imagesand renderings or particular properties collected in the database areshowing in FIG. 8A, FIG. 8B, and FIG. 8C. FIG. 8A shows a house with aconcave entrance door structure, a three step elevation, and offset withrespect to a street view. FIG. 8B shows an aligned street view of thefront entrance of the house with a five step entrance leading to thefront door. FIG. 8C is an image of an interior of a house, and includesa basement level. These images are used for training the AI engine todetermine FFH based on ROIs detected in the images that related to FFH,such as main entrance door, features around the main entrance door, suchas front step, hand railings, entrance lights, door handles, etc., aswill be discussed.

By automatically, accurately, and reliably detecting floor elevations ofproperties, the insurance industry and governments can effectivelyexpand the flood insurance market, lower financial risks from floodhazards, and increase recovery speed against flood disasters.

Methodology for Auto-Determining Floor Elevations of Properties

As illustrated in FIG. 9 , the present disclosure integrates thefollowing comprehensive information to reliably and accurately determinefloor elevations for flood insurance rating and flood risk assessment.

A building footprint dataset, contains detailed geometries of structures(like those shown in FIG. 10 ) that can be used to geospatiallycorrelate detected structure objects to the location where the objectswere detected and measured. In turn, this helps facilitate land slopeassessment along structure boundaries, to determine key land pointsaround the structure (such as lowest adjacent grade, highest adjacentgrade) that can be used in floor elevation determination, and to providethe data linkage to comprehensive information on propertycharacteristics. The polygon geometry of a building footprint containsall possible location points (such as the lowest adjacent grade point,entrance location point, and so on) around the perimeter of thebuilding. Because buildings are where people live or work, dailyactivities crossing building footprint boundaries are essential humanbehaviors. Therefore, studying the relationship between human behaviorand building structures can guide how to detect the main floor of thebuilding.

A repository of aerial photos from today's technologies (such assatellites, aircrafts, drones, and others) saved in the database oruploaded contemporaneously, help to identify walkways and other manmadelandscape features in their properties, and to discover propertyentrance points/doorstep locations. In addition, remote sensingtechnology can effectively detect manmade activities (property lighting)on the earth's surface, that can help us to automatically determine thelocations of the property entrances by identifying walkways that connectto the buildings.

Property photos are used for detecting entrance doors and doorstepsthrough AI technology and measure dimensions of detected exteriorproperty features and surrounding (such as entrance doors, doorsteps,windows, crawlspaces, basements, piers, garages, ground and others) andinterior property features (such room dimensions).

Comprehensive property information on structure characteristics (such asfoundation types, property styles, year built, basement information,structure design drawing, and so on) from property databases, is used tohelp determine or estimate first floor heights based on the engineeringdesign specifications (such as building codes) and other commoncharacteristics. In some cases, data elements for characteristics maynot be fully populated. Statistical models like those discussed hereinare used to extrapolate missing data (such as foundation types) based onproperties in neighborhoods with the information and other propertycharacteristics.

High resolution land elevation datasets generated by LiDAR technologyprovide accurate ground elevations, so that the derived first floorheights can be converted into floor elevations on a common verticaldatum. In conjunction with building footprint dataset, the conversion onderived FFHs at any grid points surrounding the structure can beconducted. Furthermore, using FFEs as a referencing elevation, thelowest floor elevations can be derived by using the dimensions ofsub-floor structures, that can be very information for flood lossassessment and insurance rate determination

Property photos including dynamic spatial information enabled by mobiledevices (such as smartphones) are used for further validating thedetection of those objects on properties. The dynamic informationcollected by smart phones, including side-mounted vehicle cameras, forexample, can effectively capture interactions between people andstructures where they are living. Many human activities can happen onthe first floor, that would lead discovering where the first floor isand what is its elevation.

The following sections describe how to reliably and accurately determineentrance locations, detect structure objects (such as doors, doorsteps,crawlspaces, basements, piers and others), correlate them to buildingfootprints, and calculate vertical heights of those objects above theground by using artificial intelligence, such as trained convolutionneural network (CNN). The AI processes described in this document(especially FIGS. 48-52 and related discussion) are used jointly forensuring the accuracy of the floor elevation determination.

Determine Property Entrance Locations from Building Footprint Geometries

Building footprint geometries are used to determine all sides of thebuildings that contain entrances to the properties by comparing thefootprint geometries with street segment geometries. Normally, theentrances of buildings are constructed to face the street networkbecause property access roads (such as driveways and walkways) need tobe connected with the road network for convenience and low cost (theshortest distance). As previously cited, U.S. Pat. No. 9,639,757describes how to obtain building footprints from images. Other buildingfootprint images are either derived from the process described in U.S.Pat. No. 9,639,757, or are already stored in the property database.

Building footprint geometries (like those shown in FIG. 10 ) may reflectsome common types of building structures, particularly residentialproperties. The shapes of some specific property designs can bereflected in their footprint geometries, which are geometric shapessaved in a digital file that can be overlaid on the perimeter of a roofline. Based on convex and concave style geometries and the size of thoseconvex and concave shapes specified in the associated buildingfootprints, the entrance locations of properties can be determined aswill be discussed in the next several figures. Furthermore, the frontentrances will be determined with reference to other points on thebuilding footprint and/or land elevations on the parcel. FIGS. 13A and13B show buildings with convex geometries, and FIG. 14 shows buildingswith concave geometries. In FIG. 14 , the walkways on the aerial photosconnect to the locations where building footprint geometry has a concavegeometry. Both convex and concave structures, in combination with theterminus of a walkway is a strong indication of a front entrance for theproperty. These are features that are used when training the AI engine,as will be discussed, to more accurately detect the FFH of a house.

With regard to multiple points around the building footprint, FIG. 11 isan image of a building footprint geometry used to determine locations ofwhere FFH is measured, defined, or referenced. As can be seen from FIG.11 , a parcel boundary (red line) is a ROI and overlaid on an arialimage. The building footprint is outlined, and a red dot is provided ata location of the footprint where the lowest adjacent grade (LAG) islocated at the perimeter of the building footprint. The lat/longitudeposition of the LAG may be known from the DEM information discussedearlier. The yellow dot on the corner of the footprint is the highestadjacent grade (HAG), and the entrance point is signified by the greenpoint. Green points are shown in adjacent properties as well, except forthe building to the left of the main building, where the location of theentrance cannot accurately be located and so it is marked with a bluedot in a middle of a street facing segment. Orange points on thefootprint signify any point where FFH is actually measured.

FIG. 12A is used to discuss in more detail a particular property with asloped terrain. FIG. 12A is a set of connected images showing, from leftto right, a left side of a house, an ariel image of the house with abuilding footprint overlay, and a front image of the house. In thisexample, the grounds slopes from front to back. The first floor, whichis part of the living space, is assumed to be flat, and thus the firstfloor nearest the street is close to the ground level, while the firstfloor overtop of the LAG could be many feet higher than ground level,depending on the severity of the ground slope. The first floor height,therefore, is a particular position (lat/long) and height above groundat the particular position. For flat properties, the FFH is consistentacross the building footprint. For sloped terrains, the FFH will varyinversely with the slope, such that the FFH will be greatest at the partof the building footprint where the ground elevation is lowest, the LAG.

In the example of FIG. 12A, the HAG, which is known from the DEM and thefootprint boundary, along with its coordinates (lat/long) is where thefront entrance is located and is at 529.0 ft. in this example. Theparcel centroid is at an elevation of 511.5 ft., and the LAG is locatedat an elevation of 521.5 ft. Coincidently, in this example, the mainentrance is offset from the left side of the house by a predetermineddistance, such as 18 ft.

Based on where the FFH's are determined/measured relative to the ground,the FFE will vary. For example, suppose the FFH is determined at theLAG. In this example, the LAG of 521.5 ft. is at a location lat/long at41.323677, −73.782870 (determined from the DEM data), and the FFH is 9.3ft above the LAG. In this situation the FFE=FFH+DEM (Lat,Long)=9.3+521.5=530.8 ft. This may be presented in a data structure as aFFH result derived from the LAG in the form of (41.323677, −73.782870,9.3 ft). On the other hand, if the FFH is determined from the propertyimagery (front image on the right side of FIG. 12A), and the FFH isdetermined to be 1.8 ft., and the position of the front entrance is at(41.323596, −73.782758), then the FFE=FFH+DEM (Lat, Long)=1.8 ft.+529ft.=530.8 ft. The FFH result extracted from a front property image maythen be presented as (41.323596, −73.782758, 1.8 ft.).

In this example, there are several different use cases for FFH, based onmeasurement points at the property. For example, in one case related tothe parcel centroid (511.5 ft.), the FFH=530.8 ft.−511.5 ft.=19.3 ft. Ina second case, relative to the building centroid, the FFH=530.8ft.−525.5 ft.=5.3 ft. In a third case where the land is flat the FFH isthe same across the property because there is not grade, and so the FFHis constant across the building footprint. However, in a fourth case,such as FIG. 12A, the ground may slope from the front of the house tothe back of the house having a walk out basement, and thus the FFH willvary with terrain elevation across the first floor.

In a situation where the first floor is not the lowest floor (i.e., thehouse has a basement), the FFH should be adjusted for floor assessmentpurposes because the basement may be flooded. For example, if the floodelevation for a 100 year flood is 527.2 ft. (i.e., the BFE=527.2 ft.)and the first floor at the lowest point of the grade adjacent to thebuilding is 3.6 ft. above the BFE, a flood premium should be chargedbecause, despite the fact that the FFH is above the BFE, the property'swalk-out basement is not. Moreover, a 100 year flood would have waterentering the basement because the first floor is only 3.6 ft. above theBFE, which means the walk-out basement (likely 9 feet below the firstfloor) is below the BFE.

In the area where land slope is relatively flat, the structures could beconstructed uniformly based on standard building codes, including thefloor. Combining digital elevation model (DEM) information and buildingfootprints, parcel boundary, or geocoding locations, relatively flatland areas can be identified based on land slope assessment on structureor near structure. With the foundation type (such as slab) informationfrom a property database, standard first floor heights (such as 10 inch)for structures on the relatively flat land areas can be assigned.

In the areas where structures sit on land slopes, the first floors couldbe elevated. The size of the elevated first floor height could becorrelated to land slopes. Therefore, land slope surrounding structurescan be used as a parameter for determining first floor heights.Specifically, the elevation difference between the LAG and the HAG alongwith building footprint geometries can be applied to build suchcorrelations. With the structure footprint dataset, elevations atdifferent points of a building footprint polygon from land elevationdatasets (such as DEM) can be extracted, so that LAG, HAG, land slope,and other land characteristics on are directly calculated as discussedabove with respect to FIG. 12A. Furthermore, parcel boundaries are usedas failover geometries for land slope characteristic assessment, in theevent that a building footprint cannot be obtained with an adequatedegree of confidence. Particularly, in many crowded urban areas, parcelboundaries can be very close to structures. Land slope assessment in anarea around the parcel centroid could reflect general characteristics(such as flat or steep) of the structures.

FIG. 12B is an AI engine training image used to train the AI engine fordetecting how FFH varies with slope.

Similarly, FIG. 12C is an AI training image used to train the AI imagefor distinguishing FFE 1201, from Freeboard/Regulation/Building CodeBFE/Flood zone 1203 level, and LAG 1205.

As mentioned above, FIGS. 13A and 13B show buildings with convexgeometries, and FIG. 14 shows buildings with concave geometries, whichare used for train the AI engine to detect main entrances on differentimages of houses.

FIG. 15 shows properties with section direction changes, and the mainentrances identified with boundary boxes detected with image processingcircuitry. These images may be taken with a street-view image capturingdevice, or service such as GOOGLE's street view, and then applied to theAI engine for detecting the main entrances of the buildings. Again,these features are used as training images to train the AI engine todetect these features to improve the probability of accuratelydetermining the FFH of a house. Commonly, entrance doors may beconstructed near where sections of the structure have significantdirection changes (such as from parallel to perpendicular to the roadnetwork). Most likely, entrances are located in the common area on thisstyle of structures, where all types of living spaces (such as kitchenand living room) connect to it. In most cases, the entrance on thecommon areas would provide convenient access.

As result, the geometries of building footprints at such locations mayhave 90 degree turns in the middle section of buildings, as demonstratedon the building footprint of FIG. 16 . The geometrical changes are usedto determine the whereabouts of property entrances. Entrance locationsof properties are identified when geometries of the detected walkwayobjects from aerial photos are connected, intersected, point to, oradjacent to the associated building footprint geometries. The list belowpresents the certainties on the entrance locations when a detectedwalkway object intersects its building footprint, and these parametersare used for learning a CNN to detect the front entrance. Walkwayobjects may be identified in specific characteristics: (1) curved andmay connect middle section of the building footprint; (2) widths ofwalkways may be much narrower than driveways and nearby roadways, and(3) walkways may have different colors from the nearby driveways androadways.

FIG. 17 are example training images of such walkways that lead to a mainentrance, as indicated by the bounding boxes around ROI.

The main entrance of a house may also be detected from the movementpattern of a mobile phone. The movement pattern is another example ofdata that is input to the AI engine to detect the location of the frontentrance of the house. The human movements can be effectively tracked bysmartphones as a common functionality of the mobile device. When amobile device (such as a smartphone) reports tracking information withthe following pattern, the location suggests entrances of theproperties: (1) the movement has a pause; and (2) the position deviceduring the pause either on a building footprint boundary or nearbuilding footprint boundary. The door can be opened from either insideor outside; and (3) the pause frequently happens at the same spots (atthe front of the door). In this situation, the homeowner can use an appthat tracks his or her activities, and the app reports to the AI enginemovement patterns for areas near and in the home. Rationale for propertyentrance detection based on human behavior/activities includes: (1) theentrance is the location of common access to the building; (2) openingor closing the entrance door take time; (3) unlocking and locking theentrance door take time; and (4) the higher the frequencies of pausedetection, the higher the certainty of the entrance detection Thelocations of the movement pauses can be automatically collected and usedfor detecting building entrance locations. When people enter or exit abuilding, they must cross the building footprint boundary line.Therefore, human behavior can be used for automatically detectingentrance locations. When a mobile device (such as a smartphone) reportswalking waypoints/walking tracks and those recorded geometries of themovements intersect with building footprint boundary geometries, thelocations could be the entrance locations of targeted properties. Thehigher the frequency of the crossing, the higher the certainty of theentrance detection.

FIG. 18 is a training image with an overlay of data tracking of asmartphone, showing provides where walking tracks converge at a commonthreshold of the house, and then diverge once inside.

Building entrances are also determined by strength changes of cellularreception signals from mobile phones or device between the inside andoutside of the building footprint boundaries. Geospatial locations ofsmall strength changes could be automatically used to determine buildingentrance locations. This too is used as additional input for learningthe AI engine.

Another method of detecting a main entrance is detecting packages nextto a house. With the online retail rapid growth, the “goods on ourdoorsteps” has become a common phenomenon. Property entrance locationsmay be identified where objects of delivered post packages are detectednear or on building footprint boundaries, if the relative position ofdelivered packages to the building corners can be measured and thepackage can be automatically georeferenced. FIG. 19A shows and exampleof this situation where packages are placed a predetermined distance(see arrow) from a boundary from the building footprint. The location ofthe packages is suggestive of the location of the main entrance.

It is doubtless that if delivered post packages can be located onbuilding footprints or near building footprints with small distances,there would be high success rates for identifying building entrancelocations when the whereabouts of delivered packages are detected aspresented on the photos below. Furthermore, many on-line companies (suchas Amazon) had started to take photos on the packages at the deliveringpoint and to prove the status of the “delivered” for their customers.The confirming photos (FIG. 19B) may provide very detailed imagery onthe bottom of entrance door, doorsteps, and locations if photos containlocation attributes. Package detection from photo images is anotherparameter that is used to learn the AI engine.

For commercial properties, vehicle frequent stopping patterns may alsobe used to help on the detection of building entrance locations. Comingand going of hotel guests, for example, could be used to analyze specialtraffic patterns for the detection of hotel entrance locations. FIG. 20is an example of this situation where cars are detected in front of ahotel entrance. Vehicle parking patters is another parameter that can beused to train the AI engine for detecting front entrances, especially oncommercial properties.

FIG. 21 is an image of a garage door and a front entrance detected in animage. A residential structure has two frequently moving parts as commoncharacteristics: house entrance door and garage door. The function ofany doors is “opening and closing”, leading visualization changes onbuildings. When buildings are monitored by a surveillance camera orsmartphone camera, for example, videos can record such changes anddetect areas of imagery changes on the structure when those doors areopening and closing. Since garage doors have much bigger sizes, smallerdoors would be the building entrances. If camera locations aregeoreferenced, the entrance locations can be determined by video data orstatic photo images before and after door opening or closing. Images ofdetected door movement is another parameter used to train the AI engine.

Determine Entrance Locations from Georeferenced Property DesignDrawings/Blueprints/Layouts

Property design drawings/blueprints/layouts are important information,that could be captured and maintained by local governments andhomeowners for many purposes, including property inspections. If thedesign drawings are georeferenced, entrance locations can be easilyidentified from the drawing and related information. Georeferencing onproperty design drawings can be achieved by matching up the corners ofthe design drawing with the corners of building footprints. Images ofproperty design drawings are another training parameter that can be usedfor training the AI engine.

In the property drawing of FIG. 22 , for example, there are two doorsymbol polygons that intersect with the building footprint boundary. Ifthe database attributes contain the information on main entrance door,the entrance can be directly defined; if not, the door facing the streetnetwork can be selected as the main entrance door of the property.

The processes described herein are jointly applied for determiningentrance locations of properties and ensuring reliability of theentrance location determination. If the building layout drawing containsdoorstep dimensions and dimensions of sub-floor rooms, the informationcan be also used directly for floor elevation determination.

Use Building Footprints to auto-Determine Highest and Lowest GroundElevations

The discussion above discussed some methods for detecting propertyentrance locations. With the building entrance location at the buildingfootprint, ground elevation (most adjacent grade) for the entrance canbe automatically determined by using the elevation dataset. Elevation ofthe lowest ground surface that touches any of the exterior walls of abuilding is LAG, which is an important parameter in flood insurancerating computation and risk assessment. LAG may indicate the highestrisk point for the structure. With the building footprint geometries,the land elevations at all boundary points can be extracted and both theLAG and the HAG can be automatically determined by comparing landelevations at all boundary points. The difference between HAG and LAGcan be also used to gauge the land slope around the structure. If thedifference is very small, it could indicate that the land at thestructure would be relatively flat; if the difference is very large, itcould represent steep land slope around the structure. FIG. 23 is animage with an overlay of LAG indicates for building structures.

Detect Entrance Doors and Doorsteps by using Property Photos and AIObject Detection

In the above sections, it was discussed how the locations of propertyentrances can be automatically detected. In property entrance locations,three important objects are used to determine FFHs. Entrance door anddoorstep are key parts of building structures, that facilitate ouraccess to the structures: main floor or first floor. As the FFHdefinition, one end of the vertical measurement is at the structure andanother end at the ground. Therefore, the ground line may need to bedetected and delineated for the height measurement. FIG. 24 shows animage with boundary boxes around steps leading to a main entrance withoverlays for ground elevation 2403 and first floor elevation 2401. InFIG. 24 the steps are used to help detect the entrance door or doorstepobjects for flood elevation determination, and to measure the height ofeither the entrance door bottom or the top of the doorstep and add theheight to the ground elevation from DEM to calculate FFEs. The dataelements for both first floor heights and FFEs are useful for a varietyof flood and insurance applications and they would be collected andstored. Thus, detection of doorsteps is another parameter used to trainthe AI engine.

With rapid enhancement on photographic capability of smartphones, thenumbers of high quality and detailed property photos were vastlyincreased during the past decade, and this provides the data needed forcapturing the entrance door, doorstep, crawlspace, basement, pier orpile and other property structure related objects. Entrance doors can bedetected by AI ROI detection (as will be discussed below, especiallywith respect to FIG. 48-52 ) based on their common characteristics (suchas shapes, colors, and styles). The vertical heights from the lowestside of an entrance door to the ground could be measured through aphotogrammetric method. The doorstep below the door object can be alsoidentified and the vertical height of the doorstep above the ground canbe determined.

FIG. 25 is an example of a front of a house with a few steps separatinga ground level from a FFE, with the height being the vertical elevationof the steps and threshold (doorstep) into the house. The measurementson those two objects can be used to determine first floor heights. Theentrance doors and doorsteps can be detected with different intelligenceand logic, and they can be used to help validate each other. FIG. 25presents a classic example of entrance door and doorstep, and heightabove ground. The ground elevation at entrance locations which areeither on or adjacent to building footprint boundaries are used forcomputing FFEs

In comparison with doorsteps, a door object contains some uniquefeatures that may give high confidence for entrance door detection, andthus used for training the AI engine to detect front entrances. Thedetected door objects of properties could be further validated by dooraccessory objects, including: door locks, including electronic locks;doorknobs; door handle sets; door knockers; door decors mailbox/slot;and other features.

FIG. 26 is an example where boundary boxes are detected around a doorknocker, a lock, and a doorknob, respectively. For a residentialproperty, door locks and doorknobs are the basic functionality ofentrance doors, that is a common requirement of functioning properties.The detected door objects containing those essential door accessoryfeatures may give very high reliability of the detection. For example,only the building main entrance would have a door knocker and mailbox.

The detected door object could be further validated between two doorlight fixtures on both side of the door objects, one nearby light, orother combination. An entrance door should have some types of lightfixtures for night access, which is a basic requirement for afunctioning property.

FIG. 27 is a training image with boundary boxes around side lights, doorknocker, and lockset/door-handle.

Every household turns-on their entrance door lights daily as a commonphenomenon. Lights can be observed from the sky. The aerial photo inFIG. 28 was taken by NASA's remote sensing technology(https://sociable.co/technology/nasas-black-marble-image-shows-how-stunning-the-earth-is-at-night/.)that captures city lights at night over Europe. The technology cancapture images on nights with or without moonlight, producing crispviews of the earth's atmosphere, land and ocean surfaces. When lightlocations detected by such technology are near or on buildingfootprints, they would give high possibility to identify the locationsof property entrances. Integrating the space technology and buildingfootprint dataset is another technological way to determine propertyentrance locations. As an alternative technology, the lights at nightcan be also detected by airborne drones. The logic for using buildingfootprint data to filter the entrance light locations can be applied todrone technology. Similarly, detected lights that belong to specificbuildings can be used for detecting main entrances. For example,streetlights separate from the lights that were mounted on the buildings(on building footprints) or very close to the buildings could be used toassist in detecting at least the intended front of the building.

A detected entrance door object can be validated by a doorbell objectnext to it. Some doorbells have a self-lighting functionality, that is auseful and unique feature for the doorbell object detection. As itsunique purpose of use, a doorbell must be installed near the entrancedoor. The building door object could be detected or validated when theobject is near a badge reader for commercial buildings. The securitylock with keypad also belongs to this category.

The door object could be detected or validated within two hand-railobjects. See the boundary boxes in FIG. 29 showing the detection ofhandrails, side mounted lights, and door handle. It is one commonfeature that entrance doors could have handrails, particularly elevatedproperties. Some elevated properties could have significant hand-railsand stairs from the ground to the first floor deck or entrance. Thedetection on hand-rail objects can be used to extract the lowestelevation for the entire property enclosure. The entrance objects may bedetected by typical or common style shapes (such as arches, tringles,squares, and others) and designs, that may be more complex and uniquethan other parts of the buildings.

Directly detecting doorsteps is another effective method to determinethe FFH and the entrance locations of properties, and thus used as aparameter for training the AI engine. Doorsteps have commoncharacteristics, patterns, and styles, in which each step has an equalheight and they will reach the bottom of the entrance door where thefirst floor is located. With the detected doorstep objects, heights ofthe doorstep can be measured through a photogrammetry method. The lowestdoorstep should align with the ground, where the ground elevation can beextracted from a DEM dataset. By adding doorstep height on the top ofground elevations, FFEs can be determined. Again, both first floorheights (doorstep height) and FFEs are collected and stored by thistechnology as data elements.

The detected objects on delivered post packages can exactly sit on boththe entrance doors and FFEs (as in FIG. 19A). The bottoms of thosepackage objects represent another way to determine where the top of thedoorstep or the bottom of the door are. If the height of the bottom ofthe package above the ground can be measured, it would be the firstfloor height. This method demonstrates how human activities are used bythe AI engine to project FFH/FFEs. The property entrance is also theentrance for home goods, which is one of the most important function ofproperty entrances, utilized by this technology.

When a high-accuracy and sufficient elevation measurement device (e.g.,an altimeter), is available, simply deliver or place the device on theproperty entrance or directly on the first floor or lowest floor, andthe flood elevation can be measured. The device can be a one-time use orrental device.

The first floor height could be also measured if crawlspace objects canbe detected and heights of crawlspaces above the ground measured, suchas the crawlspaces detected with boundary boxes in FIG. 30 . Combiningthe crawlspace height and the ground elevation, the FFE is directlyderived. Properties can be elevated with a crawlspace against flooding,which is one common approach for mitigating flood inundation risk.

The heights of crawlspaces can be determined by using object detectionand photogrammetric methods, extracting from interior crawlspace photos.Commonly, crawlspace photos were collected through property inspectionprocesses, such as that shown by the boundary box in FIG. 31 .

Many properties have sub-floor windows, that indicated the structuresare partially under the ground. Since the properties are partiallyunderground, the lowest floor elevation can be simply determined byexterior photos, with additional measurements on interior structures orextrapolation logic. When sub-floor window objects were detected, thelowest floor elevation can be estimated by: determining ground elevationfrom digital elevation management (DEM, like that available from thenational elevation dataset, NED); adding the lowest window heights; andsubtracting the height measurement to basement floor or a standardheight below basement window. First floor heights could vary withdifferent types of foundation types (such as Slab, Crawl Space, CrawlSpace—Unexcavated, Basement—below Grade, Basement—Daylight,Basement—Walkout, Piers, Hillside, and others). The foundation typeinformation from a property database can be used as one key parametersin first floor height determination. Standard first floor heights withadditional modifiers specified in this application can be applied to theproperties in the nation.

With the information, an estimate of the lowest floor elevation iscalculated. FIG. 32A shows an image of a standard height of a basementwindow, and its height relative to a basement floor. FIG. 32B shows animage with a related basement window flush with the ground, while FIG.32C shows a basement window detected with an offset height relative tothe ground. The similar method can be also applied to detect objectsthat lead to identify the walkout basement.

When a vehicle is detected below the building entrance, most likely, thestructures were elevated; when the vertical space below the buildingentrance is open, height support objects (such as poles) for thestructure can be used to determine first floor heights. FIG. 33 shows anexample of boundary boxes around vehicles below a house. Otherstructures (an air conditioner on the left, and a box on the right) arealso detected as regions of interest in FIG. 33 . By adding the groundelevation into the height of support objects, the first floor/lowestfloor elevation can be determined.

When a property was elevated, associated utility units are be elevatedto the same elevation where the building platform is too. If a utilityunit (such as an air-conditioning unit, see boundary box on the left inFIG. 33 ) near the building footprint is detected and its height issignificantly above ground, the heights of the utility units can be usedas supporting a physical fact for the first floor determination.

The building construction types, foundation types and floor typeinformation, for example, can be used to estimate first floor heightsbased on the design specifications on those components. An examplefoundation type code is presented in FIG. 40 . Furthermore, materialsfor constructing foundations can be used for enhancing the FFHcalculation because of their structure engineering characteristics.Foundational type information may not be available for all properties inthe US. A statistical model is used based on similarities in propertycharacteristics and terrain conditions from nearby neighborhoods forextrapolating missing foundation information. Aggregated statisticaldata from a radius area of properties with known foundation types, or atZIP code & ZIP Plus 4 level can be useful as an alternative approach.Building codes (such as design floor elevation) could vary withdifferent geographical areas, particularly political regions (such asstates and counties) and land areas (such as inland and coastal areas).Therefore, geographic variations on building codes can be considered inthe first floor height determination. The national database for buildingcodes and design floor elevation or height can be developed forfacilitating FFH computation. Some information on the structurespecifications could be collected and validated through home inspectionand appraisal process, such as the CoreLogic database that includesappraisal information.

The garage type (such as basement garage, first floor garage, and so on)in the property information database can be used to estimate the firstfloor heights by using garage dimension information. Garage positionrelevant to the building entrance can be also measured by using AIobject detection technology from property photos. Garage doors normallyhave standard dimensions for one, two, and three cars. In addition, ifthe driveway object can be extracted from aerial photos, the drivewaywill connect to garage doors.

Detailed garage information in the property database can be very usefulfor deciding garaging positions and dimensions, so that the first floorand lowest floor elevations can be derived. The example informationrelated to garages is listed FIG. 39 .

Machinery and equipment objects can be also detected by the objectdetection technology. FIG. 34 shows the detected boiler, furnace, andair-conditioning components. If the floor where those detected equipmentobjects sit is determined, lowest floor elevation can be assigned to theequipment, so that they can be used on flood damage assessment. Theinformation could be extracted from the property database or propertyinspection reports, which may indicate the floor where utilities wereinstalled.

The first floor height could also vary with year built, structure typesand changes on construction cost, materials, technologies with the time.Many modern building codes may not be implemented in old days.Therefore, age of buildings/time built could be factored into the firstfloor height calculation. For example, a structure built in 50's couldhave a significant first height comparison with new constructions.Therefore, the year built information on the structures is an importantattribute for the FFH determination.

The first floor elevation was also regulated by flood elevations ifstructures were built in Special Flood Hazard Areas (SFHAs), defined bythe federal government. Sizes of additional freeboard above the BaseFlood Elevation (BFE) and other considerations from builders also affectthe first floor heights. Therefore, the vertical relationship betweenBFE and LAG at the structure is valuable information for projecting howhigh the structure should be elevated if the structure was built afterthe initial date that FEMA flood insurance rate map (FIRM) data becameeffective. For example, the first floor was constructed on the top ofthe garage for a property in the oceanfront, that could elevate thefirst floor above BFE+freeboard and create more usable space. Those dataelements described above were fully used in the FFH projection model forconsidering federal, state and local flood regulations on floodmitigation. The information from Community Rating System (CRS) formitigating flood risk can be considered as fact on the first floorelevation determination. Hence, the Pre-FIRM and Post-FIRM conditionshave their influence of first floor heights because not all propertieswere elevated, depending on when the original FEMA map panels wereissued.

Determination of Lowest Floor Elevations by Using Photogrammetry onInterior Property Photos and Property Database

The most important objective on floor elevation determination for floodrisk assessment is the lowest floor elevations, where would have thedeepest water inundation. The insurance companies and governments mayrequire to directly use the lowest elevation or indirectly derive themfrom FFEs. The lowest floor elevations may not be directly detected fromexterior property photos if properties contain sub-floors, the floorsbelow FFEs. Therefore, interior property photos and comprehensivedatabases for property characteristics would be needed for theassessment, like that shown in FIG. 35 , showing a subterranean basementlevel 3501 only accessible from a set of stairs to the main level 3503.

It is to be noted that FFEs can be the lowest floor elevations ifunderground structures do not exist. When the window objects aredetected and their positions below house entrance, the properties couldhave a split level or basement. When properties are detected withsub-floors, the process for determining the lowest floor elevations hasto be conducted. For this case, the height of sub-floor below the firstfloor needs to be obtained.

The lowest floor elevation can be determined by using derived FFE andmeasured basement dimension abstracted from basement photos, like thatin FIG. 36 . Lowest floor elevation=FFE−Sub-floor Height

Referencing to the basement, the FFE becomes the next higher floorelevation. Providers may use different terminology in the insurancepricing. The lowest floor elevation can be determined by using derivedFFE and the derived floor height from photos on the split-level styleproperties, like that shown in FIG. 37 .

Split-level style properties with sub-floors can also be identified whenthe ground elevation at the building entrance is substantially higherthan the elevations at other points on the building footprint boundary.For example, the property presented in the training image of FIG. 38 haselevations on the right side of the building which are much higher thanon the left side of the building, where LAG is located. Therefore, theelevation variance between the lowest point and the highest point on thebuilding footprint (difference between LAG and HAG) can be used toauto-detect the split-level style properties when the elevation varianceexceeds a certain threshold.

Split-level style properties and properties with basements can be alsoidentified by using the property information database that is associatedwith building footprints and property parcels. Example story codes forsplit style properties are listed in FIG. 41 .

The lowest floor elevations can be derived by subtracting heights forthose sub-floor components (such as basement) from FFEs/FFHs. Specificheights could directly come from the property database or could beestimated by typical design specifications.

Many new communities may be built uniformly in terms of their designspecifications on the foundation types. Previously determined floorheight specifications above the ground for some individual propertiescan be applied to the same community when property design specificationsare matching. For example, if houses built in a street block or anentire community have a uniform 0.75 ft. doorstep height, FFEs for otherbuildings in the neighborhood could be equal to 0.75 ft.+groundelevation at any targeted property. The method is to try using thesimilarity on property information for effective floor elevationdetermination. The results from this similarity method could bevalidated by the methods described in previous sections.

Using Building Footprint Data and Side-Mounted Original Vehicle Camerato Capture Doorstep Height/FFEs

As described above, acquiring elevation certificates for properties isexpensive. Some expensive equipment & technologies (such as LiDAR) weretried to apply to capture the doorstep height/FFEs by shooting lights atdoorsteps from driving vehicles. The LiDAR technology, for example,sends light to buildings and receives reflected light to determinedistances to the targets and other measurements. Because of the cost ofsuch survey technology, it was not broadly used for the propertyinformation collection.

In this disclosure a new method is described regarding how to useordinary cameras (such as from smartphones) to determine doorstepheights. The core difference is to use building footprint datasets tofacilitate distance computation between the targeted buildings andmoving vehicles without requiring the equipment for receiving reflectionfrom the targeted objects (such as doorsteps). With the new method, thecost for acquiring floor elevation determination would be significantlyreduced.

In above sections, methods were discussed for automatically detectingentrance locations. The entrance door point at a building footprintboundary and GPS point at the driving vehicle give the distance betweenthe vehicle and the targeted building. The distance between two pointscan be used to automatically compute the scale factor between the objecton the photo and actual object, so that expensive equipment relying onlight reflection is not needed. FIG. 42 shows an example of this wherethe vehicle is on a roadway at a known height with a known GPS point onthe vehicle. Likewise, the distance to the property from the vehicle isknown. With using the building footprint data, any passing vehicles withregular cameras (such as cameras from smartphones) can capture theinformation. The high resolution DEM (such as 1-meter USGS DEM) can bealso applied to ensure the accuracy of elevation measurements.

Determining Floor Elevations by Geospatially Enabling Digital ContentsInside Properties

With today's AI object detection technique and digital video & photos,the contents inside properties could be detected and recognized. Sincethose detected objects are contained within the building footprints,those content objects can be automatically georeferenced by correlatingthem with boundary points (on walls) on building footprints throughphotogrammetric measurements. As demonstrated in FIG. 43 , thenortheastern corner of building has the real-world coordinates(latitude, longitude). By adding the distance offset to the buildingcorner where the dishwasher is located, for example, the real-worldcoordinates would be derived and assigned to the dishwasher (redboundary box) inside the building. The procedure can be applied for alldetected contents within the building footprints, so that all detectedcontent objects can be georeferenced and mapped. Commercially, mapping acertain brand of dishwashers in a geography, for example, could haveprofound commercial values:

-   -   Map geographic distributions of certain types of home goods        inside properties    -   Map geographic distributions of certain brands of home goods        inside properties    -   Map geographic distributions of quantity of home goods inside        properties    -   Map geographic distributions of ages of home goods inside        properties    -   Map geographic distributions of styles of furniture arrangements        inside properties    -   Map geographic distributions of home interior design and        upgrades inside properties    -   Map geographic distributions of overall home content values        inside properties

Geometric relationship between the entrance location and the corner ofthe structure on the building footprint can be measured from the insideof the structure by using smartphones with proximity sensor and othermeasurement tools. The methods can be also applied to geospatiallyenable all contents (such as appliances, furniture, and so on) in theinside of the structure.

The floor plan of structures (like that shown in FIG. 22 ) can begeoreferenced to line up with building footprint geometries. After afloor plan is referenced, detailed dimension measurements from thestructure design can be used to facilitate geospatial correlation, forexample, between entrance door and building corners. CoreLogic haddeveloped a technology to generate accurate floor plan from 3D videodata.

When an elevation measurement enabled device 4401 is placed on the topof detected objects (such as a countertop), the floor elevation can bedetermined by subtracting the height of the object (see FIG. 44 ). Theelevation enabled devices can be:

-   -   A mail letter or a package is sent to the address of the        properties, that contain the device has the technology that        accurately determine elevations on a common vertical datum.        -   The device can be for one-time use.        -   The device can be a rental.    -   Smart phones, that may be enhanced with advanced elevation        technology.    -   Smart watches, that may be enhanced with advanced elevation        technology.    -   Smart eyewear, that may be enhanced with advanced elevation        technology.

Furthermore, the detect objects inside building footprints includepeople. Today, technology has reached the capability for human facerecognition. Therefore, phone devices in human hands, watches on thearms, and eyeglasses on faces from persons in properties would berelatively easy objects to be detected and relative heights above thefloor can be measured in a reliable manner. For example, when readingelevation data from the smartphone, the height above the ground can bemeasured by using photos or videos, so that elevation from the phonescan be corrected to the floor level.

If the flood inundation photos are available, dimension changes ofdetected content objects with and without water inundation can be usedfor directly and accurately calculating water depth above floorelevation inside inundated properties. The auto-detected water depthscan be applied for flood insurance claims and flood loss computation. Ingeneral, after the content objects inside properties are digitallycaptured, object changes from the impacts and interruptions of naturalhazard events can be used for hazard loss assessments.

First floor height could vary with future conditions of natural hazardimpacts. The chart below shows NOAA observations on global sea levelrise(https://www.climate.gov/news-features/understanding-climate/climate-change-global-sea-level)since 1880 (See FIG. 45 ). The FFH can be designed based on futureconditions of the sea level rise. Therefore, the climate model forprojecting sea level rise can be considered as a component for FFHdetermination.

The first floor heights can vary with many property characteristics. Thecorrelations between first floor heights and property characteristicscould be established by conducting statistical analysis on the nationalproperty database with millions of property records and comprehensiveproperty attributes, including appraisal and MLS databases. The imagepresented below shows rich information captured by CoreLogic technology,that can be used for FFH determination too. FFH=Function (PropertyCharacteristics, Land Characteristics)

Construction costs, affordability, community styles can be additionalfactors in FFH determinations. For example, a luxury property could haveits own category and customized specifications on the foundation and FFHdesign. Those economic factors could be considered in FFH computationand modeling. CoreLogic has reconstruction cost data for all propertiesin the United States, that can be directly used in the FFH modeling.

Traditionally method for determining FFHs through expensive land surveyswas only referencing to the lowest adjacent grade. This innovatedlocation based FFH methodology allows AI technology to extract FFHs fromany angles and any directions (such as front, back, side, as well asfrom the roof too), having a great flexibility and adoptabilityTraditionally method for determining FFHs through expensive land surveyswas only referencing to the lowest adjacent grade. This innovatedlocation based FFH methodology allows AI technology to extract FFHs fromany angles and any directions (such as front, back, side, as well asfrom the roof too), having a great flexibility and adoptability of thenew technology. There are many fenced houses and communities in thecountry. For those properties, only property-front imageries could beavailable. Therefore, it is important to have the flexibility for FFHextraction of the new technology.

There are many fenced houses and communities in the country. For thoseproperties, only property-front imageries could be available. Therefore,it is important to have the flexibility for FFH extraction.

This disclosure has developed a significant amount of intelligences todetect targeted objects of properties and facilitate floor elevationdeterminations. The floor elevations (such as the FFEs and lowest floorelevations) can be measured, derived, and validated by multiple methodsfor ensuring the reliability of the measurements. Building footprintgeometries play a very important role to ensure correctness andefficiency of the floor elevation determination. The technology useshuman behavior (timing and frequency) crossing building footprintboundaries for automatically determining locations of property entrance,that bring a new dimension into the object detection technology.

The technology enabling georeferenced content objects within buildingfootprints have great commercial market values, assigning real worldcoordinates to the detected home contents inside properties. As today,99.9% of home contents have no GPS capability. With this technology,they will receive reliable real world geocodes, making them mappable.Geospatially enabling the contents inside structures can effectivelyhelp to flood damage assessment on the content because we knowwhereabouts of the contents.

Because high resolution elevation dataset and property photos and videoare used in the process, the desired precision of elevation measurementscan be reached, so that costs on manual elevation surveys andcertificate acquisitions can be saved. Dimensional changes on previouslydetected content objects within properties from photos with flood waterinundations can be used to determine flood depth for insurance claimsand flood loss validations. As result, costs for flood risk assessmenton properties and flood insurance policy purchases can be effectivelyreduced. The technology can be used to promote growth of flood insurancemarket and the reduction of financial impacts when future flood disasteroccurs. Fundamentally, the technology can significantly increase theaccuracy of flood loss estimation of analytical models.

Application of AI Engine

AI plays an important role in discovering and identifying the housestructure and substructure. Semantic segmentation method will be used toidentify at a pixel level, what is a door, what is part of these steps,and what is potential structure below the assumed first floor, i.e. thedoor. Labeling a large number of property photos at pixel levels andbuild training data comprising of doors, doorsteps, patios, exteriorwalls, etc. based on comprehensive collection of property photos andtraining data comprising doors, staircases, etc. Using the data toconstruct an AI model for the first floor height extraction.

The surrounding components of street concrete or lawn/garden are alsoused for the first floor height determination. There are a number ofdifferent methods to collect this information.

As discussed with respect to FIG. 42 , the first is to triangulate basedon additional data such as a street view, and try to extract based onknown door heights and measured directly in the image where we cancorrelate pixels to inches. As an option, it is possible to locate thefront door or first floor, on the digital elevation model, whichcombined with the footprint should give us an idea of the elevation ofthe surrounding property of the house. A machine learning model is usedthat allows to identify common door and other structure heights toestimate at first floor height, in addition to sub first floor.

There are two ways to accomplish approach. The first is to identify eachof the pieces separately, and try combine them at the end. This might bea brute force approach of recording all of the steps, that is to sayidentify door pixels, step pixels, sub floor pixels, estimate each ofthe heights individually based on known norms in the region, and providethose. A second approach is to take a full model approach, where a setof ground truth (GT) outputs are provided for the model, and train intoend solution which is capable of determining the entire set of desiredoutputs within a deep learning model. Such approaches have been shown tobe the most effective ways to model multiple sensors, or data inputs.This approach will be discussed in more detail below. This would allowthe process to take advantage of available data sets, such asCorelogic's extensive property database when making determinations, andnot relying solely on what is available in the visual image.

FIG. 46 is an image of a front of a house with sematic segmentation,identifying regions of interest the forms of a basement window, astaircase, and a front door. DSM (digital surface model) generated froma lidar scan is used to create a ground truth for the height of thebuilding and the surrounding grounds. Once the depth map is extractedfrom the DSM, a CNN is trained to predict new depth maps. These depthmaps are then used to measure heights from building surfaces and theground. The pre-existing imageries can be aligned to 3D measurements onthe structures and land surface for calculating relative dimensions ofdetected objects and deriving the first floor heights.

With the technology, FFHs can be also defined by referencing roofelevations too, that is another innovated way to for FFH determination.In this approach, ground elevation may not be necessarily needed. If thebuilding height was used for FFH extraction, the location of the heightmeasurement will be captured. Roofs of buildings may be flat either,particularly true for majority of residential properties. Therefore,location-based FFH determination is very important.

Interior imageries, room dimensional measurements, 3D room design modelsand other interior property information can be also conjunctionally usedwith information described above for determining first floor heights andlowest floor elevation (such as basement). For a single floor structure,for example, if the building height and elevation are available, thefirst floor elevation can be estimated by subtracting the room heightfrom building/roof elevation.

The present disclosure may be embodied as a system, a method, and/or acomputer program product. The computer program product may include acomputer readable storage medium on which computer readable programinstructions are recorded that may cause one or more processors to carryout aspects of the embodiment.

The derived first floor heights can be validated by property imagery,aerial photos, street view tools, and site visiting and surveys. Forexample, floor information documented in Elevation Certificates can beused for the validation purpose. The AI based FFH measurement can beconsidered as observation/physical data, it would be used to replace FFHvalues derived from other method (such as from foundation type).

FIG. 47 is a diagram of related resources available to for determiningthe FFH/FFE of buildings in particular properties and recording them ina data structure, along with a vast array of other buildings for otherparticular properties. The ultimate resource is a database of FFH/FFEinformation for particular properties, as shown in 4719, which includesAddress, City, State ZIP, (FFH, Lat/Lon), (FFE, Lat/Lon), (LAG,Lat/Lon), lowest floor elevation (LFE), Foundation Type, andReconstruction Cost, for each property. This information is thenaccessible for homeowners, insurance underwriters, appraisers and thelike for assessing the possibility of flood damage for the particularproperty.

Tabular property databases 4701 are available from Corelogic whichprovide vast volumes of building/property specific information forproperties across the US. In addition to real estate specific databases,such as Diablo, and Interchange, which provide building featurecharacteristics, and building reconstruction costs for individualproperties, other databases are available such as Corelogic's appraisaldatabase that provide property characteristic data and cost data to helpassess cost damage due to the extent of flood damage as discussed above.The building cost information may be obtained via recorded informationor calculated from an automated valuation model (AVM), such asCorelogic's AVM(https://www.corelogic.com/landing-pages/automated-valuation-models.aspx).The data from these databases, and AVM are merged with foundation typeinformation 4703 also stored in Corelogic databases such as ZIP4 andKnowledge table (example foundation information is also shown in FIG. 40) in 4705, and then an initial FFH assignment is made in 4707, withinput from adjustments made based on information in 4709 regard flooddata, land slope, building footprint, and DEM, as well as a testingdataset converted from an elevation certificate, if available. Thisresults in a standard format for the initial foundation assignment in4707. This initial FFH is then adjusted by AI engine input in 4713,which combines the data from 4707 with property image data collectedfrom multiple listing service photos and Streetview API 4711. This AIadjusted FFH is provided to 4715, where it is combined with buildingfootprint and DEM data from 4717 to provide Geospatially enabled FFH in4715, which in turn produces the data structures shown in 4719.

The methods and systems described herein may be implemented usingcomputer programming or engineering techniques including computersoftware, firmware, hardware or any combination or subset thereof,wherein the technical effects may include at least one of: system andmethod that is configured to use ground truth images of features of abuilding to train a CNN, and apply the trained to new images todetermine a FFE in a property. Examples of Moreover, variousphotogrammetric methods are described herein that estimate from photosentrance doors, doorsteps and other features at a residence that suggestthe location of a FFE. Various ground truth photos are used to train theCNN with features detected in the photo, such as a door. A rule used intraining the CNN is deriving a scale factor by comparing a number ofpixels on the detected object (e.g., a door) in the photo against aground truth photo with a known door size (in absolute terms and in apixel space) as well as an actual standard door dimension.

Hereinafter, how a computing device 100 calculates a main floor height,which is a vertical distance in the real world, between a ground and alower boundary of an entrance door, will be explained.

First, by referring to FIG. 48 , a configuration of the computing device100 will be explained.

The computing device 100 may include a data extraction network 200 and adata analysis network 300. Further, to be illustrated in FIG. 50 , thedata extraction network may include at least one first featureextracting layer 210, at least one Region-Of-Interest (ROI) poolinglayer 220, at least one first outputting layer 230 and at least one datavectorizing layer 240. Also illustrated in FIG. 51 , the data analysisnetwork 300 may include at least one second feature extracting layer 310and at least one second outputting layer 320.

Below, specific processes of calculating the main floor height will beexplained. To begin with, a first embodiment of the present disclosurewill be presented.

First, the computing device 100 may acquire at least one subject image.By referring to FIG. 49 , the subject image may correspond to a scene ofa subject house, photographed from a front of the subject house,including an image of a main entrance door of the subject house and animage of a ground below the subject house.

After the subject image is acquired, in order to generate a sourcevector to be inputted to the data analysis network 300, the computingdevice 100 may instruct the data extraction network 200 to generate thesource vector including (i) an apparent height, which is a distancebetween the ground and a lower boundary of the main entrance door on thesubject image, and (ii) an apparent size, which is a size of the mainentrance door on the subject image.

In order to generate the source vector, the computing device 100 mayinstruct at least part of the data extraction network 200 to detect themain entrance door and the ground on the subject image.

Specifically, the computing device 100 may instruct the first featureextracting layer 210 to apply at least one first convolutional operationto the subject image, to thereby generate at least one subject featuremap. Thereafter, the computing device 100 may instruct the ROI poolinglayer 220 to generate one or more ROI-Pooled feature maps by poolingregions on the subject feature map, corresponding to ROIs on the subjectimage which have been acquired from a Region Proposal Network (RPN)interworking with the data extraction network 200. Furthermore, thecomputing device 100 may instruct the first outputting layer 230 togenerate at least one estimated ground location and at least oneestimated main entrance door location. That is, the first outputtinglayer 230 may perform a classification and a regression on the subjectimage, by applying at least one first Fully-Connected (FC) operation tothe ROI-Pooled feature maps, to generate each of the estimated groundlocation and the estimated main entrance door location, includinginformation on coordinates of each of bounding boxes. Herein, thebounding boxes may include the ground and the main entrance door.

After such detecting processes are completed, by using the estimatedground location and the estimated main entrance door location, thecomputing device 100 may instruct the data vectorizing layer 240 tosubtract a y-axis coordinate of an upper bound of the ground from ay-axis coordinate of the lower boundary of the main entrance door togenerate the apparent height, and multiply a vertical height of the mainentrance door and a horizontal width of the main entrance door togenerate the apparent size.

After the apparent height and the apparent size are acquired, thecomputing device 100 may instruct the data vectorizing layer 240 togenerate at least one source vector including the apparent height andthe apparent size as its at least part of components.

Then, the computing device 100 may instruct the data analysis network300 to calculate an estimated main floor height by using the sourcevector. Herein, the second feature extracting layer 310 of the dataanalysis network 300 may apply second convolutional operation to thesource vector to generate at least one source feature map, and thesecond outputting layer 320 of the data analysis network 300 may performa regression, by applying at least one FC operation to the sourcefeature map, to thereby calculate the estimated main floor height.

As shown above, the computing device 100 may include two neuralnetworks, i.e., the data extraction network 200 and the data analysisnetwork 300. The two neural networks should be trained to perform saidprocesses properly. Below, how to train the two neural networks will beexplained by referring to FIG. 50 and FIG. 51 .

First, by referring to FIG. 50 , the data extraction network 200 mayhave been trained by using (i) a plurality of training imagescorresponding to scenes of subject houses for training, photographedfrom fronts of the subject houses for training, including images oftheir corresponding main entrance doors for training and images of theircorresponding grounds for training, and (ii) a plurality of theircorresponding GT ground locations and GT main entrance door locations.More specifically, the data extraction network 200 may have appliedaforementioned operations to the training images, and have generatedtheir corresponding estimated ground locations and estimated mainentrance door locations. Then, (i) each of ground pairs of each of theestimated ground locations and each of their corresponding GT groundlocations and (ii) each of door pairs of each of the estimated mainentrance door locations and each of the GT main entrance door locationsmay have been referred to, in order to generate at least one height lossand at least one door loss, by using any of loss generating algorithms,e.g., a smooth-L1 loss algorithm and a cross-entropy loss algorithm.Thereafter, by referring to the height loss and the door loss,backpropagation may have been performed to learn at least part ofparameters of the data extraction network 200. Parameters of the RPN canbe trained also, but a usage of the RPN is a well-known prior art, thusfurther explanation is omitted.

Herein, the data vectorizing layer 240 may have been implemented byusing a rule-based algorithm, not a neural network algorithm. In thiscase, the data vectorizing layer 240 may not need to be trained, and mayjust be able to perform properly by using its settings inputted by amanager.

As an example, the first feature extracting layer 210, the ROI poolinglayer 220 and the first outputting layer 230 may be acquired by applyinga transfer learning, which is a well-known prior art, to an existingobject detection network such as VGG or ResNet, etc.

Second, by referring to FIG. 51 , the data analysis network 300 may havebeen trained by using (i) a plurality of source vectors for training,including apparent heights for training and apparent sizes for trainingas their components, and (ii) a plurality of their corresponding GT mainfloor height. More specifically, the data analysis network 300 may haveapplied aforementioned operations to the source vectors for training, tothereby calculate their corresponding estimated main floor heights fortraining. Then each of height pairs of each of the estimated main floorheights and each of their corresponding GT main floor heights may havebeen referred to, in order to generate at least one height loss, byusing said any of loss algorithms. Thereafter, by referring to theheight loss, backpropagation can be performed to learn at least part ofparameters of the data analysis network 300.

After performing such training processes, the computing device 100 canproperly calculate the estimated main floor height by using the subjectimage including the scene photographed from the front of the subjecthouse.

Hereafter, other embodiments will be presented.

A second embodiment is similar to the first embodiment, but differentfrom the first embodiment in that the source vector thereof furtherincludes a tilt angle, which is an angle between an optical axis of acamera which has been used for photographing the subject image (e.g.,the subject house) and a vertical axis of the ground, as its additionalessential component. Also, in order to calculate the tilt angle to beincluded in the source vector, the data extraction network of the secondembodiment may be slightly different from that of the first one. Inorder to use the second embodiment, it should be assumed thatinformation on a principal point and focal lengths of the camera areprovided.

Specifically, in the second embodiment, the data extraction network 200may have been trained to further detect lines of a road in the subjectimage, to thereby detect at least one vanishing point of the subjectimage. Herein, the lines of the road may denote lines representingboundaries of the road located on the ground in the subject image, andthe vanishing point may denote where extended lines generated byextending the lines of the road, which are parallel in the real world,are gathered. As an example, through processes performed by the firstfeature extracting layer 210, the ROI pooling layer 220 and the firstoutputting layer 230, the lines of the road may be detected.

After the lines of the road are detected, the data vectorizing layer 240may find at least one point where the most extended lines are gathered,and determine it as the vanishing point. Thereafter, the datavectorizing layer 240 may calculate the tilt angle by referring toinformation on the vanishing point, the principal point and the focallengths of the camera by using a following formula:

θ_(tilt) =a tan 2(vy−cy,fy)

In the formula, vy may denote a y-axis coordinate of the vanishingpoint, cy may denote a y-axis coordinate of the principal point, and fymay denote a y-axis focal length. Using such formula to calculate thetilt angle is a well-known prior art, thus more specific explanation isomitted.

After the tilt angle is calculated, the data vectorizing layer 240 mayset the tilt angle as a component of the source vector, and the dataanalysis network 300 may use such source vector to calculate theestimated main floor height. In this case, the data analysis network 300may have been trained by using the source vectors for trainingadditionally including tilt angles for training.

As a third embodiment, the source vector may further include an actualdistance, which is a distance in a real world between the camera and themain entrance door, as an additional component of the source vector. Forthe third embodiment, it is assumed that a camera height, which is adistance between the camera and a ground directly below the camera inthe real world, is provided. This embodiment is same as the secondembodiment until the first outputting layer 230 generates the tiltangle. Hereinafter, processes performed after the tilt angle isgenerated will be explained.

The computing device 100 may instruct the data analysis network 300 tocalculate the actual distance by referring to information on the cameraheight, the tilt angle, a coordinate of the lower boundary of the mainentrance door, by using a following formula:

$d_{actual} = \sqrt{\begin{matrix}{{\frac{h^{2} + {h^{2}\tan^{2}\left\{ {\frac{\pi}{2} + \theta_{tilt} - {{atan}\left( \frac{y - {cy}}{fy} \right)}} \right\}}}{1 + \frac{\left( {y - {cy}} \right)^{2}}{{fy}^{2}}}\left( \frac{x - {cx}}{fx} \right)^{2}} +} \\{h^{2}\tan^{2}\left\{ {\frac{\pi}{2} + \theta_{tilt} - {{atan}\left( \frac{y - {cy}}{fy} \right)}} \right\}}\end{matrix}}$

In the formula, x and y may denote coordinates of the lower boundary ofthe main entrance door, fx and fy may denote the focal lengths for eachaxis, cx and cy may denote coordinates of the principal point, and h maydenote the camera height. A usage of such formula for calculating theactual distance is a well-known prior art, thus further explanation isomitted.

After the actual distance is calculated, the data vectorizing layer 240may set the actual distance as the additional component of the sourcevector, and the data analysis network 300 may use such source vector tocalculate the estimated main floor height. Also, in this case, the dataanalysis network 300 may have been trained by using the source vectorsfor training additionally including actual distances for training.Similarly, as discussed with respect to FIG. 42 , with known cameraheight h, and known distance to the property from a street view, anelevation of the entry point (front door) of the building is directlydetermined by identifying the height of the front steps. Assuming thesteps are vertical, the triangle shown in FIG. 42 becomes a righttriangle. The distance to the steps, d, is known from above, and anangle, θ, that opposes the height of the stairs is measured from thecamera image. Accordingly, the height, h, is determined as tan (θ)*d=h.

For a fourth embodiment which is mostly similar to the first one, someinformation acquired from a subject house DB storing information onsubject houses, including the subject house, can be used for generatingthe source vector. That is, the computing device 100 may acquirestructure information on a structure of the subject house, e.g., 3bedrooms, 2 toilets, 1 basement, etc., from the subject house DB. Or,the computing device 100 may acquire topography information on atopography of a region around the subject house, e.g., hill, flat,mountain, etc., from the subject house DB. Herein, at least one of thestructure information and the topography information can be added to thesource vector by the data vectorizing layer 240, and the data analysisnetwork 300, which has been trained by using the source vectors fortraining additionally including corresponding information, i.e., atleast one of the structure information and the topography information,may use such source vector to calculate the estimated main floor height.

As a fifth embodiment, the source vector, generated by using any of thefirst to the fourth embodiments, can be concatenated channel-wise to thesubject image or its corresponding subject segmented feature map, whichhas been generated by applying an image segmentation operation thereto,to thereby generate a concatenated source feature map, and the dataanalysis network 300 may use the concatenated source feature map tocalculate the estimated main floor height. An example configuration ofthe concatenated source feature map may be shown in FIG. 52 . In thiscase, the data analysis network 300 may have been trained by using aplurality of concatenated source feature maps for training including thesource vectors for training, other than using only the source vectorsfor training. By using the fifth embodiment, much more information canbe inputted to processes of calculating the estimated main floor height,thus it can be more accurate. Herein, if the subject image is useddirectly for generating the concatenated source feature map, it mayrequire too much computing resources, thus the subject segmented featuremap may be used for reducing a usage of the computing resources.

Descriptions above are explained under an assumption that the subjectimage has been photographed from the front of the subject house,however, embodiments stated above may be adjusted to be applied to thesubject image photographed from other sides of the subject house. Andsuch adjustment will be easy for a person in the art, referring to thedescriptions.

In reference to FIG. 53 , the property entrance location (latitude andlongitude) is captured based on the street facing segment of thebuilding footprint geometries and street network geometries. Theentrance location can be determined by the method presented below:

-   -   Create the centroid of the building footprint on a targeted        property: Point 1    -   From Point 1, find the nearest point at the street network with        the same street name of the property: Point 2.    -   Determine the intersecting point between the line from Point 1        and Point 2 and the building footprint boundary: Point 3. Point        3 is the middle point of the street-facing segment on the        building footprint.    -   Use Point 3 and Point 1 to divide the building footprint        geometry into 4 quadrants.    -   From Point 1, find the farthest point of the building footprint        boundary in each quadrant that Point 3 is touched: Point 4 and        Point 5, that would be the street-facing corners of the        building. The line segment on the building footprint between        Point 4 and Point 5 will be the street-facing segment.    -   If the distance between the entrance and a building corner is        measured by using AI/imagery technology or any other method, the        coordinates (Latitude, Longitude) of the entrance location can        be calculated as Point 6.

The enhance location can be used to facilitate simple tape measurementor auto-measurements from smartphones with proximity sensors on thedoorstep height from the ground or photo measurement in a close distancerange. The measurements from the methods described above can also beused for capturing premixes between detected objects (such as entrancedoors, doorsteps) and points (such as building corners) of buildingfootprints. If the distance measurement from the property entrance andbuilding corners are not available, the middle point (point 3 in themethod described above) of the street-facing segment of buildingfootprints can be used. Locations associated with LAG and HAG can bedetermined by using building footprints and digital elevation model.

Tiered accuracy rules can be developed for determining the confidencelevels of different approaches as summarized in FIG. 54 .

The follow description involving FIG. 55 relates to the computingenvironment and circuitry used perform the operations discussed herein.

The computer readable storage medium may be a tangible device that canstore instructions for use by an instruction execution device(processor). The computer readable storage medium may be, for example,but is not limited to, an electronic storage device, a magnetic storagedevice, an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any appropriate combination of thesedevices. A non-exhaustive list of more specific examples of the computerreadable storage medium includes each of the following (and appropriatecombinations): flexible disk, hard disk, solid-state drive (SSD), randomaccess memory (RAM), read-only memory (ROM), erasable programmableread-only memory (EPROM or Flash), static random access memory (SRAM),compact disc (CD or CD-ROM), digital versatile disk (DVD) and memorycard or stick. A computer readable storage medium, as used in thisdisclosure, is not to be construed as being transitory signals per se,such as radio waves or other freely propagating electromagnetic waves,electromagnetic waves propagating through a waveguide or othertransmission media (e.g., light pulses passing through a fiber-opticcable), or electrical signals transmitted through a wire.

Computer readable program instructions described in this disclosure canbe downloaded to an appropriate computing or processing device from acomputer readable storage medium or to an external computer or externalstorage device via a global network (i.e., the Internet), a local areanetwork, a wide area network and/or a wireless network. The network mayinclude copper transmission wires, optical communication fibers,wireless transmission, routers, firewalls, switches, gateway computersand/or edge servers. A network adapter card or network interface in eachcomputing or processing device may receive computer readable programinstructions from the network and forward the computer readable programinstructions for storage in a computer readable storage medium withinthe computing or processing device.

Computer readable program instructions for carrying out operations ofthe present disclosure may include machine language instructions and/ormicrocode, which may be compiled or interpreted from source code writtenin any combination of one or more programming languages, includingassembly language, Basic, Fortran, Java, Python, R, C, C++, C# orsimilar programming languages. The computer readable programinstructions may execute entirely on a user's personal computer,notebook computer, tablet, or smartphone, entirely on a remote computeror compute server, or any combination of these computing devices. Theremote computer or compute server may be connected to the user's deviceor devices through a computer network, including a local area network ora wide area network, or a global network (i.e., the Internet). In someembodiments, electronic circuitry including, for example, programmablelogic circuitry, field-programmable gate arrays (FPGA), or programmablelogic arrays (PLA) may execute the computer readable programinstructions by using information from the computer readable programinstructions to configure or customize the electronic circuitry, inorder to perform aspects of the present disclosure.

Aspects of the present disclosure are described herein with reference toflow diagrams and block diagrams of methods, apparatus (systems), andcomputer program products according to embodiments of the disclosure. Itwill be understood by those skilled in the art that each block of theflow diagrams and block diagrams, and combinations of blocks in the flowdiagrams and block diagrams, can be implemented by computer readableprogram instructions.

The computer readable program instructions that may implement thesystems and methods described in this disclosure may be provided to oneor more processors (and/or one or more cores within a processor) of ageneral purpose computer, special purpose computer, or otherprogrammable apparatus to produce a machine, such that the instructions,which execute via the processor of the computer or other programmableapparatus, create a system for implementing the functions specified inthe flow diagrams and block diagrams in the present disclosure. Thesecomputer readable program instructions may also be stored in a computerreadable storage medium that can direct a computer, a programmableapparatus, and/or other devices to function in a particular manner, suchthat the computer readable storage medium having stored instructions isan article of manufacture including instructions which implement aspectsof the functions specified in the flow diagrams and block diagrams inthe present disclosure.

The computer readable program instructions may also be loaded onto acomputer, other programmable apparatus, or other device to cause aseries of operational steps to be performed on the computer, otherprogrammable apparatus or other device to produce a computer implementedprocess, such that the instructions which execute on the computer, otherprogrammable apparatus, or other device implement the functionsspecified in the flow diagrams and block diagrams in the presentdisclosure.

FIG. 55 is a functional block diagram illustrating a networked system800 of one or more networked computers and servers. In an embodiment,the hardware and software environment illustrated in FIG. 55 may providean exemplary platform for implementation of the software and/or methodsaccording to the present disclosure.

Referring to FIG. 55 , a networked system 800 may include, but is notlimited to, computer 805, network 810, remote computer 815, web server820, cloud storage server 825 and compute server 830. In someembodiments, multiple instances of one or more of the functional blocksillustrated in FIG. 55 may be employed.

Additional detail of computer 805 is shown in FIG. 55 . The functionalblocks illustrated within computer 805 are provided only to establishexemplary functionality and are not intended to be exhaustive. And whiledetails are not provided for remote computer 815, web server 820, cloudstorage server 825 and compute server 830, these other computers anddevices may include similar functionality to that shown for computer805.

Computer 805 may be a personal computer (PC), a desktop computer, laptopcomputer, tablet computer, netbook computer, a personal digitalassistant (PDA), a smart phone, or any other programmable electronicdevice capable of communicating with other devices on network 810.

Computer 805 may include processor 835, bus 837, memory 840,non-volatile storage 845, network interface 850, peripheral interface855 and display interface 865. Each of these functions may beimplemented, in some embodiments, as individual electronic subsystems(integrated circuit chip or combination of chips and associateddevices), or, in other embodiments, some combination of functions may beimplemented on a single chip (sometimes called a system on chip or SoC).

Processor 835 may be one or more single or multi-chip microprocessors,such as those designed and/or manufactured by Intel Corporation,Advanced Micro Devices, Inc. (AMD), Arm Holdings (Arm), Apple Computer,etc. Examples of microprocessors include Celeron, Pentium, Core i3, Corei5 and Core i7 from Intel Corporation; Opteron, Phenom, Athlon, Turionand Ryzen from AMD; and Cortex-A, Cortex-R and Cortex-M from Arm.

Bus 837 may be a proprietary or industry standard high-speed parallel orserial peripheral interconnect bus, such as ISA, PCI, PCI Express(PCI-e), AGP, and the like.

Memory 840 and non-volatile storage 845 may be computer-readable storagemedia. Memory 840 may include any suitable volatile storage devices suchas Dynamic Random Access Memory (DRAM) and Static Random Access Memory(SRAM). Non-volatile storage 845 may include one or more of thefollowing: flexible disk, hard disk, solid-state drive (SSD), read-onlymemory (ROM), erasable programmable read-only memory (EPROM or Flash),compact disc (CD or CD-ROM), digital versatile disk (DVD) and memorycard or stick.

Program 848 may be a collection of machine readable instructions and/ordata that is stored in non-volatile storage 845 and is used to create,manage and control certain software functions that are discussed indetail elsewhere in the present disclosure and illustrated in thedrawings. In some embodiments, memory 840 may be considerably fasterthan non-volatile storage 845. In such embodiments, program 848 may betransferred from non-volatile storage 845 to memory 840 prior toexecution by processor 835.

Computer 805 may be capable of communicating and interacting with othercomputers via network 810 through network interface 850. Network 810 maybe, for example, a local area network (LAN), a wide area network (WAN)such as the Internet, or a combination of the two, and may includewired, wireless, or fiber optic connections. In general, network 810 canbe any combination of connections and protocols that supportcommunications between two or more computers and related devices.

Peripheral interface 855 may allow for input and output of data withother devices that may be connected locally with computer 805. Forexample, peripheral interface 855 may provide a connection to externaldevices 860. External devices 860 may include devices such as akeyboard, a mouse, a keypad, a touch screen, and/or other suitable inputdevices. External devices 860 may also include portablecomputer-readable storage media such as, for example, thumb drives,portable optical or magnetic disks, and memory cards. Software and dataused to practice embodiments of the present disclosure, for example,program 848, may be stored on such portable computer-readable storagemedia. In such embodiments, software may be loaded onto non-volatilestorage 845 or, alternatively, directly into memory 840 via peripheralinterface 855. Peripheral interface 855 may use an industry standardconnection, such as RS-232 or Universal Serial Bus (USB), to connectwith external devices 860.

Display interface 865 may connect computer 805 to display 870. Display870 may be used, in some embodiments, to present a command line orgraphical user interface to a user of computer 805. Display interface865 may connect to display 870 using one or more proprietary or industrystandard connections, such as VGA, DVI, DisplayPort and HDMI.

As described above, network interface 850, provides for communicationswith other computing and storage systems or devices external to computer805. Software programs and data discussed herein may be downloaded from,for example, remote computer 815, web server 820, cloud storage server825 and compute server 830 to non-volatile storage 845 through networkinterface 850 and network 810. Furthermore, the systems and methodsdescribed in this disclosure may be executed by one or more computersconnected to computer 805 through network interface 850 and network 810.For example, in some embodiments the systems and methods described inthis disclosure may be executed by remote computer 815, computer server830, or a combination of the interconnected computers on network 810.

Data, datasets and/or databases employed in embodiments of the systemsand methods described in this disclosure may be stored and or downloadedfrom remote computer 815, web server 820, cloud storage server 825 andcompute server 830.

FIG. 56 is a flowchart of how the AI engine described in reference toFIGS. 48-52 operates. While the flowchart and AI engine have beendescribed with respect to determining FFH, it should be understandingthat other features, such as FFE, main entrance location, grade ofterrain, overlap of BFE with respect to FFH/FFE, housing model (e.g.,colonial, or split-level), whether there is a subfloor, crawlspace, orelevation are features that are also part of the present description.For training the AI engine, images and described herein may be used astraining images and GTs to isolate the ROIs and determine the featuresfor the parameters sought.

The process begins in step S560 where an analysis of a particularproperty beings. The footprint of the building on the property isretrieved, detected, or determined as discussed earlier. Other propertycharacteristic data for the particular property is obtained in S562,using databases in 4701, and other resources in 4703 and 4705 in FIG. 47. An initial FFH is made in S564 (4707 in FIG. 47 ) and in step S566adjustments are made to the FFH based on EC information, land slope, DEMand other property specific information from 4709 (FIG. 47 ) so the FFHis associated with the DEM for the parcel of land on which the buildingis located. As part of the adjustments, in step S568 a query is maderegarding whether the first floor has a floor beneath it (e.g., abasement). If so, the adjustment is made to FFH to account for thesub-floor (e.g., basement) in step S570, and then process proceeds tostep S572 where the data structure of FFH like that shown in 4707 (FIG.47 ) is saved. However, if the response to the query in step S568 isnegative, the process proceeds directly to step S572.

After the initial FFH is set, the process optionally proceeds to provideinput to an end user. In the part of the process flow, the processproceeds to a query in step S574 where the FFE is compared to BFE dodetermine if BFE is above FFE (recognizing the FFE and FFH are directlyrelated). If the response to the query is negative, then there is noimmediate action taken because the FFH is above the base flood level,and so a graphic is prepared that shows BFE below FFE for the particularbuilding in step S576. On the other hand, if the response to the queryin affirmative, the process proceeds to step S578, where an indicatorindicating that the BFE is above the FFE is included in the graphic soas to alert an end user that there is an elevated risk of flood damagefor that particular property.

FIG. 57A is a flowchart the corresponds with the training of the dataextraction network of the AI engine as previously discussed with respectto FIGS. 48-52 . The process begins in step S5760 where training images(like those shown in the various figures of this document) are appliedto a features extraction layer where features are detected in theimages, such as the bounding boxes showing the in the figures of thisdocument. The process then proceeds to step S5762 where ground truth(GT) images are input to the data extraction network in step S5762. Thenin step S5764 estimates are generated for the detected features, and instep S5766 losses are generated for the extracted features, with respectto the GTs, and backpropagated so as to learn the data extractionparameters of the data extraction network.

FIG. 57B is a flowchart that corresponds with the training of the dataanalysis network of the AI engine as previously discussed with respectto FIGS. 48-52 . The process begins in step S5768 where a trainingvector is input with respect to apparent features as well ascorresponding vectors that are GTs. In step S5770 the losses for theparameters are determined by comparison, and then in step S5772 thelosses are back-propagated so as to learn the data analysis parametersof the data analysis network.

Modifications, additions, or omissions may be made to the systems,apparatuses, and methods described herein without departing from thescope of the disclosure. For example, the components of the systems andapparatuses may be integrated or separated. Moreover, the operations ofthe systems and apparatuses disclosed herein may be performed by more,fewer, or other components and the methods described may include more,fewer, or other steps. Additionally, steps may be performed in anysuitable order. As used in this document, “each” refers to each memberof a set or each member of a subset of a set.

To aid the Patent Office and any readers of any patent issued on thisapplication in interpreting the claims appended hereto, applicants wishto note that they do not intend any of the appended claims or claimelements to invoke 35 U.S.C. 112(f) unless the words “means for” or“step for” are explicitly used in the particular claim.

The descriptions of the various embodiments of the present inventionhave been presented for purposes of illustration, but are not intendedto be exhaustive or limited to the embodiments disclosed. Manymodifications and variations will be apparent to those of ordinary skillin the art without departing from the scope and spirit of the describedembodiments. The terminology used herein was chosen to best explain theprinciples of the embodiments, the practical application or technicalimprovement over technologies found in the marketplace, or to enableothers of ordinary skill in the art to understand the embodimentsdisclosed herein.

The computer readable storage medium may be a tangible device that canstore instructions for use by an instruction execution device(processor). The computer readable storage medium may be, for example,but is not limited to, an electronic storage device, a magnetic storagedevice, an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any appropriate combination of thesedevices. A non-exhaustive list of more specific examples of the computerreadable storage medium includes each of the following (and appropriatecombinations): flexible disk, hard disk, solid-state drive (SSD), randomaccess memory (RAM), read-only memory (ROM), erasable programmableread-only memory (EPROM or Flash), static random access memory (SRAM),compact disc (CD or CD-ROM), digital versatile disk (DVD) and memorycard or stick. A computer readable storage medium, as used in thisdisclosure, is not to be construed as being transitory signals per se,such as radio waves or other freely propagating electromagnetic waves,electromagnetic waves propagating through a waveguide or othertransmission media (e.g., light pulses passing through a fiber-opticcable), or electrical signals transmitted through a wire.

Computer readable program instructions described in this disclosure canbe downloaded to an appropriate computing or processing device from acomputer readable storage medium or to an external computer or externalstorage device via a global network (i.e., the Internet), a local areanetwork, a wide area network and/or a wireless network. The network mayinclude copper transmission wires, optical communication fibers,wireless transmission, routers, firewalls, switches, gateway computersand/or edge servers. A network adapter card or network interface in eachcomputing or processing device may receive computer readable programinstructions from the network and forward the computer readable programinstructions for storage in a computer readable storage medium withinthe computing or processing device.

Computer readable program instructions for carrying out operations ofthe present disclosure may include machine language instructions and/ormicrocode, which may be compiled or interpreted from source code writtenin any combination of one or more programming languages, includingassembly language, Basic, Fortran, Java, Python, R, C, C++, C# orsimilar programming languages. The computer readable programinstructions may execute entirely on a user's personal computer,notebook computer, tablet, or smartphone, entirely on a remote computeror computer server, or any combination of these computing devices. Theremote computer or computer server may be connected to the user's deviceor devices through a computer network, including a local area network ora wide area network, or a global network (i.e., the Internet). In someembodiments, electronic circuitry including, for example, programmablelogic circuitry, field-programmable gate arrays (FPGA), or programmablelogic arrays (PLA) may execute the computer readable programinstructions by using information from the computer readable programinstructions to configure or customize the electronic circuitry, inorder to perform aspects of the present disclosure.

Aspects of the present disclosure are described herein with reference toflow diagrams and block diagrams of methods, apparatus (systems), andcomputer program products according to embodiments of the disclosure. Itwill be understood by those skilled in the art that each block of theflow diagrams and block diagrams, and combinations of blocks in the flowdiagrams and block diagrams, can be implemented by computer readableprogram instructions.

The computer readable program instructions that may implement thesystems and methods described in this disclosure may be provided to oneor more processors (and/or one or more cores within a processor) of ageneral purpose computer, special purpose computer, or otherprogrammable apparatus to produce a machine, such that the instructions,which execute via the processor of the computer or other programmableapparatus, create a system for implementing the functions specified inthe flow diagrams and block diagrams in the present disclosure. Thesecomputer readable program instructions may also be stored in a computerreadable storage medium that can direct a computer, a programmableapparatus, and/or other devices to function in a particular manner, suchthat the computer readable storage medium having stored instructions isan article of manufacture including instructions which implement aspectsof the functions specified in the flow diagrams and block diagrams inthe present disclosure.

The computer readable program instructions may also be loaded onto acomputer, other programmable apparatus, or other device to cause aseries of operational steps to be performed on the computer, otherprogrammable apparatus or other device to produce a computer implementedprocess, such that the instructions which execute on the computer, otherprogrammable apparatus, or other device implement the functionsspecified in the flow diagrams and block diagrams in the presentdisclosure.

Obviously, numerous modifications and variations of the presentinvention are possible in light of the above teachings. It is thereforeto be understood that within the scope of the appended claims, theinvention may be practiced otherwise than as specifically describedherein.

1. A method of detecting a first floor height (FFH) of a first floor ofa subject building, the method comprising: extracting digital surfacemodel (DSM) information from a database, the DSM information includessurface elevation information of the subject building on a parcel ofland on which the subject building is located; applying an overheadimage of the subject building to a CNN-based AI engine, which haspreviously been trained, so as to identify a first floor of the subjectbuilding from the image, the CNN-based AI engine having previously beentrained with other images of a plurality of other buildings, the otherimages including overhead images of the plurality of other buildings;analyzing the image with the CNN-based AI engine to estimate the FFH ofthe subject building, the analyzing including detecting a roof of thesubject building from the overhead image, identifying a roof elevationof the roof of the subject building from the DSM information, anddetermining an interior height differential between the roof elevationand the first floor of the subject building; and estimating the FFH ofthe subject building as a difference between the roof elevation and theheight differential between the roof elevation and first floor.
 2. Themethod of claim 1, wherein the other images previously applied to theCNN-based AI engine include interior images of interior spaces of theplurality of other buildings.
 3. The method of claim 2, wherein theanalyzing includes the CNN-based AI engine identifying regions ofinterest (ROIs) in the interior images of the interior spaces of theplurality of other buildings.
 4. The method of claim 3, wherein the ROIsinclude at least one of an appliance, a countertop, or a piece offurniture.
 5. The method of claim 3, wherein the ROIs includes aninterior structure.
 6. The method of claim 5, wherein the interiorstructure includes at least one of an interior staircase, an entrance, adoor, or a doorstep.
 7. The method of claim 1, further comprising:extracting previously stored data from another database that includesinterior property information for the subject building.
 8. The method ofclaim 7, wherein the another database is a multiple listing service(MLS) database.
 9. The method of claim 7, wherein the interior propertyinformation includes at least one of room dimensional measurements,number of floors, foundation type, property type, basement information,structure design drawing, or a 3D design model.
 10. The method of claim1, further comprising: training the CNN-based AI engine by inputtingother images as training images and ground truth images to the CNN-basedAI engine, and backpropagating losses so as to establish data extractionparameters for a data extraction network portion of the CNN-based AIengine.
 11. A method of detecting a first floor height (FFH) of a firstfloor of a subject building, the method comprising: applying an image ofthe subject building to a CNN-based AI engine, which has previously beentrained, so as to identify a first floor of the subject building fromthe image, the CNN-based AI engine having previously been trained withother images of a plurality of other buildings, the other imagesincluding at least one of a front, a side, a roof, or a back-side viewof individual buildings of the plurality of other buildings; analyzingthe image with the CNN-based AI engine and determining the FFH of thesubject building; extracting digital surface model (DSM) informationfrom a database, the DSM information includes surface elevationinformation of the terrain and/or surface elevation information from adataset for the parcel of land on which the subject building is located;converting the FFH of the subject building to a first floor elevation(FFE) from the FFH and the DSM information; and identifying a height ofthe subject building and the FFE of the subject building based on theheight of the subject building.
 12. The method of claim 11, furthercomprising: obtaining information on a building footprint of the subjectbuilding; and identifying a location at an adjacent grade point alongthe building footprint so as to determine an elevation of the locationand the FFE of the subject building at the location.
 13. The method ofclaim 11, further comprising: training the CNN-based AI engine, thetraining including inputting other images as training images and groundtruth images to the CNN-based AI engine, and backpropagating losses soas to establish data extraction parameters for a data extraction networkportion of the CNN-based AI engine.
 14. The method of claim 13, whereinthe training further comprises: training the CNN-based AI engine todetect building types including applying training images of differenttypes of buildings to the CNN-based AI engine, the different types ofbuildings including basement-below grade, basement-daylight, andbasement-walkout.
 15. The method of claim 13, wherein the trainingfurther comprises: training the CNN-based AI engine to detect foundationtypes by applying training images of buildings with different types offoundations.
 16. The method of claim 13, wherein the training of theCNN-based AI engine further includes training the CNN-based AI enginewith overhead images of buildings having roofs and walkways leading tothe main entrance.
 17. The method of claim 12, wherein the obtainingincludes determining a geometry of the building footprint from aperimeter of a roof line of the subject building.
 18. The method ofclaim 13, wherein the training further comprises: training the CNN-basedAI engine to estimate a height of a main entrance of the subjectbuilding, and the image being a street view image of the subjectbuilding, wherein the image includes an object detected in the image asat least one of steps and a ramp leading to the main entrance, and theheight of the main entrance of the subject building is estimated from anestimated height of the objected detected in the image.
 19. The methodof claim 11, further comprising: extracting previously stored data inanother database that includes interior property information for thesubject building.
 20. The method of claim 19, wherein the interiorproperty information includes at least one of room dimensionalmeasurements, number of floors, foundation type, property type, basementinformation, structure design drawing, or a 3D design model.